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Abstract. In this paper, we consider the message forwarding problem 
that consists in managing the network resources that are used to for- 
ward messages. Previous works on this problem provide solutions that 
either use a significant number of buffers (that is n buffers per proces- 
sor, where n is the number of processors in the network) making the 
solution not scalable or, they reserve all the buffers from the sender to 
r) . the receiver to forward only one message . The only solution that uses a 

Q constant number of buffers per link was introduced in pQ. However the 

solution works only on a chain networks. In this paper, we propose a 
CZ3 , snap-stabilizing algorithm for the message forwarding problem that uses 

^ | ■ the same complexity on the number of buffers as [T] and works on tree 

topologies. 
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VO ' 1 Introduction 

f^) ' It is known that the quality of a distributed system depends on its fault tol- 

erance. Many fault-tolerance approaches have been introduced, for instance: 
Self-Stabilization [J] which allows the conception of systems that are tolerant 
of any arbitrary transient fault. A system is said to be self-stabilizing if start- 
ing from any arbitrary configuration, the system converges into the intended 

r^j ' behavior in a finite time. Another instance of the fault-tolerance scheme is the 

!w , snap-stabilization [5] . Snap-stabilizing systems always behave according to their 

specification, and this regardless of the starting configuration. Thus a snap- 
stabilizing solution can be seen as a self-stabilizing solution that stabilizes in 
zero time. 

In distributed systems, the end-to-end communication problem consists in 
delivery in finite time across the network of a sequence of data items generated 
at a node called the sender, to another node called the receiver. This problem 
comprises the following two sub-problems: (?) the routing problem, i.e., the de- 
termination of the path followed by the messages to reach their destinations; 
(ii) the message forwarding problem that consists in the management of net- 
work resources in order to forward messages. In this paper, we focus on the 
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second problem whose aim is to design a protocol that manages the mechanism 
allowing the message to move from a node to another one on the path from a 
sender to a receiver. Each node on this path has a reserved memory space called 
buffer. With a finite number of buffers, the message forwarding problem consists 
in avoiding deadlock and livelock situations. 

The message forwarding problem has been well investigated in a non faulty 
setting |4I5I6I7| . In |8I9] self-stabilizing solutions were proposed. Both solutions 
deal with network dynamic, i.e., systems in which links can be added or removed. 
However, they assume that the routing tables are correct (loop- free). Thus the 
proposed solutions cannot ensure absence of deadlocks or message loss during 
the stabilization time. 

In this paper, we address the problem of providing a snap-stabilizing protocol 
for this problem. Snap-stabilization provides the desirable property of delivering 
to its recipient every message generated after the faults, once and only once even 
if the routing tables are not (yet) stabilized. Some snap-stabilizing solutions have 
been proposed to solve the problem |10lllll) . In [TU], the problem was solved 
using n buffers per node (where n denotes the number of processors in the 
network). The number of buffers was reduced in |llj to D buffers per node 
(where D refers to the diameter of the network). However, the solution works by 
reserving the entire sequence of buffers leading from the sender to the receiver. 
Note that the first solution is not suitable for large-scale systems whereas the 
second one has to reserve all the path from the source to the destination for the 
transmission of only one message. In [I] , a snap-stabilizing solution was proposed 
using a constant number of buffers per link. However the solution works only on 
chain topologies. 

We provide a snap-stabilizing solution that solves the message forwarding 
problem in tree topologies using the same complexity on the number of buffers 
as in [1^ i.e., 25 + 1 buffers by processor, where 5 is the degree of the processor 
in the system. 

Road Map The rest of the paper is organized as follow: Our Model is presented 
in Section [2] In Section |3l we provide our snap-stabilizing solution for the mes- 
sage forwarding problem. The proofs of correctness are given in Sub-Section 13.31 
Finally we conclude the paper in Section 0J 

2 Model and Definitions 

Network. We consider in this paper a network as an undirected connected graph 
G = (V, E) where V is the set of nodes (processors) and E is the set of bidirec- 
tional communication links. Two processors p and q are said to be neighbours 
if and only if there is a communication link (p,q) between the two processors. 
Note that, every processor is able to distinguish all its links. To simplify the 
presentation we refer to the link (p, q) by the label q in the code of p. In our case 
we consider that the network is a tree of n processors. 



Computational model. In this paper we consider the classical local shared 
memory model introduced by Dijkstra [12] known as the state model. In this 



model communications between neighbours are modelled by direct reading of 
variables instead of exchange of messages. The program of every processor con- 
sists in a set of shared variables (henceforth referred to as variable) and a finite 
number of actions. Each processor can write in its own variables and read its 
own variables and those of its neighbours. Each action is constituted as follow: 

< Label >::< Guard > — > < Statement > 

The guard of an action is a boolean expression involving the variables of 
p and its neighbours. The statement is an action which updates one or more 
variables of p. Note that an action can be executed only if its guard is true. Each 
execution is decomposed into steps. 

The state of a processor is defined by the value of its variables. The state of 
a system is the product of the states of all processors. The local state refers to 
the state of a processor and the global state to the state of the system. 

Let y € C and A an action of p (p € V). A is enabled for p in y if and 
only if the guard of A is satisfied by p in y. Processor p is enabled in y if 
and only if at least one action is enabled at p in y. Let P be a distributed 
protocol which is a collection of binary transition relations denoted by — », on 
C . An execution of a protocol P is a maximal sequence of configurations e = 
yoyi—ViVi+i--- such that, V i > 0, y t ->• y i+ \ (called a step) if y t+1 exists, 
else yi is a terminal configuration. Maximality means that the sequence is either 
finite (and no action of P is enabled in the terminal configuration) or infinite. 
All executions considered here are assumed to be maximal. £ is the set of all 
executions of P. Each step consists on two sequential phases atomically executed: 
(i) Every processor evaluates its guard; (ii) One or more enabled processors 
execute its enabled actions. When the two phases are done, the next step begins. 
This execution model is known as the distributed daemon |13j . We assume that 
the daemon is weakly fair, meaning that if a processor p is continuously enabled, 
then p will be eventually chosen by the daemon to execute an action. 

In this paper, we use a composition of protocols. We assume that the above 
statement (ii) is applicable to every protocol. In other words, each time an 
enabled processor p is selected by the daemon, p executes the enabled actions of 
every protocol. 

Snap- Stabilization. Let r be a task, and Sr a specification of r . A protocol 
P is snap-stabilizing for Sp if and only if VT 6 £, r satisfies Sp- 



Message Forwarding Problem. The message forwarding problem is specified 
as follows: 



Specification 1 (SP) A protocol P satisfies SP if and only if the following 
two requirements are satisfied in every execution of P: (i) Any message can be 
generated in a finite time, (ii) Any valid message is delivered to its destination 
once and only once in a finite time. 



Buffer Graph A Buffer Graph [T3] is defined as a directed graph on the buffers 
of the graph i.e., the nodes are a subset of the buffers of the network and links are 
arcs connecting some pairs of buffers, indicating permitted message flow from one 
buffer to another one. Arcs are only permitted between buffers in the same node, 
or between buffers in distinct nodes which are connected by a communication 
link. 



3 Message Forwarding 

In this section, we first give an overview of our snap stabilizing Solution for the 
message forwarding problem, then we present the formal description followed by 
some sketches of the proofs of correctness. 



3.1 Overview of the Solution 

In this section, we provide an informal description of our snap stabilizing solu- 
tion that solves the message forwarding problem and tolerates the corruption of 
the routing tables in the initial configuration. We assume that there is a self- 
stabilizing algorithm that calculates the routing tables and runs simultaneously 
to our algorithm. We assume that our algorithm has access to the routing tables 
via the function Next p (d) which returns the identity of the neighbour to which p 
must forward the message to reach the destination d. In the following we assume 
that there is no message in the system whose destination is not in the system. 

Before detailing our solution let us define the buffer graph used in our solu- 
tion: 

Let 6(j>) be the degree of the processor p in the tree structure. Each processor 
p has (?) one internal buffer that we call Extra buffer denoted EXT p . (ii) S(p) 
input buffers allowing p to receive messages from its neighbors. Let q £ N p , the 
input buffer of p connected to the link (p,q) is denoted by IN p (q). (Hi) 8{p) 
output buffers allowing it to send messages to its neighbors. Let q £ N p , the 
output buffer of p connected to the link (p,q) is denoted by OUT p {q). In other 
words, each processor p has 28(j>) + 1 buffers. The generation of a message is 
always done in the output buffer of the link (p, q) so that, according to the routing 
tables, q is the next processor for the message in order to reach its destination. 

The overall idea of the algorithm is the following: When a processor wants 
to generate a message, it consults the routing tables to determine the next 
neighbour by which the message will transit in order to reach its destination. 
Once the message is on system, it is routed according to the routing tables: 
Let us refer to nb(m, b) as the next buffer b' of the message m stored in b, 
b e {IN p (q) V OUT p (q)}, q G N p . We have the following properties: 

1. nb(m, IN p (q)) = OUT p (q') such as q' is the next process by which m has to 
transit to reach its destination. 

2. nb(m,OUT p (q)) = IN q {p) 

Thus, if the message m is in the Output buffer OUT p (q) such as p is not the 
destination then it will be automatically copied in the Input buffer of q. If the 



the message m is in the Input buffer of p (IN p (q)) then if p is not the destination 
it consults the routing tables to determine which is the next process by which 
the message has to pass in order to meet its destination. 

Note that when the routing tables are stabilized and when all the messages 
are in the right direction, the first property nb(m,IN p (q)) = OUT p (q') is never 
verified for q — q' . However, this is not true when the routing tables are not yet 
stabilized and when some messages are in the wrong direction. 

Let us now recall the message progression. A buffer is said to be free if and 
only if it is empty (it contains no message) or contains the same message as 
the input buffer before it in the buffer graph buffer. In the opposite case, a 
buffer is said to busy. The transmission of messages produces the filling and 
the cleaning of each buffer, i.e., each buffer is alternatively free and busy. This 
mechanism clearly induces that free slots move into the buffer graph, a free slot 
corresponding to a free buffer at a given instant. 

In the following, let us consider our buffer graph taking in account only active 
arcs (an arc is said to be active if it starts from a non empty buffer). Observe 
that in this case the sub graph introduced by the active arcs can be seen as 
a resource allocation graph where the buffers correspond to the resources, for 
instance if there is a message m in IN p (q) such as nb(m,IN p (q)) = OUT q i(p) 
then m is using the resource (buffer) IN p (q) and it is asking for another resource 
which is the output buffer OUT p (q'). In the following we will refer to this sub 
graph as the active buffer graph. 

It is known in the literature that a deadlock situation appears only in the case 
there exists a cycle in the resource allocation graph. Note that this is also the case 
in our active buffer graph. Observe that because our buffer graph is built on a tree 
topology, if a cycle exists then we are sure that there are at least two messages 
m and m' that verifies the following condition: nb(m,IN p (q)) = OUT p (q) A 
nb(m' , IN p i(q')) — OUT p i(q'). Since in this paper we consider a distributed 
system, it is impossible for a processor p to know whether there is a cycle in the 
system or not if no mechanism is used to detect them. The only thing it can do 
is to suspect the presence of a cycle in the case there is one message in its input 
buffer IN p (q) that has to be sent to OUT p (q). In order to verify that, p will 
initiate a token circulation that will follow the active buffer graph starting from 
the input buffer containing the message m. By doing so, the token circulation 
either finds a free buffer (refer to Figure [TJ (b)) or detects a cycle. Note that two 
kinds of cycle can be detected: (i) a Full-Cycle involving the first input buffer 
containing m (refer to Figure [TJ (a)) or (ii) a Sub-Cycle that does not involve 
the input buffer that contains the message m (refer to Figure [TJ (c)). 

If the token circulation has found an empty buffer (Let refer to it as B), 
the idea is to move the messages along the token circulation path to make the 
free slot initially on B move. By doing so, we are sure that OUT p (q) becomes 
free. Thus p can copy the message m directly to OUT p (q) (Note that this action 
has the priority on all the other enabled actions). If the token circulation has 
detected a cycle then two sub-cases are possible according to the type of cycle 
detected: (i) The case of a Full-Cycle: Note that in this case p is the one that 
detects the cycle. The aim will be to release OUT p (q). (ii) The case of a Sub- 
Cycle: In this case the processor containing the last buffer B that is reached 
by the token is the one that detects the cycle (Processor q in Figure [TJ (c)). 





(a) Instance of a Full-Cycle. 



(b) Free Buffer on the path 
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(c) Instance of a Sub-Cycle. (d) Token circulations deadlocked 

Fig. 1. Instance of token circulations. 



Note that B is an input buffer. The aim in this case is to release the output 
buffer B' by which the message m in B has to be forwarded to in order to meet 
its destination {OUT q (r) in Figure [TJ (c)). Note that B 1 is in this case part of 
the path of the token circulation. In both cases (i) and (ii), the processor that 
detects the cycle copies the message from the corresponding input buffer (either 
from IN p (q) or B) to its extra buffer. By doing so the processor releases its 
input buffer. The idea is to move messages on the token circulation's path to 
make the free slot that was created on the input buffer move. This ensures that 
the corresponding output buffer will be free in a finite time (either OUT p (q) or 
B 1 ). Thus the message in the extra buffer can be copied in the free slot on the 
output buffer. Thus one cycle has been broken. 

Note that many token circulations can be executed in parallel. To avoid 
deadlock situations between the different token circulations (refer to Figure [TJ 
(d)), the token circulation with an identifier id can use a buffer of another token 
circulation having the identifier id' if id < id' . Note that by doing so, one token 
circulation can break the path of another one when the messages move to escort 
the free slot. The free slot can be then lost. For instance, in Figure [5J we can 
observe that the free slot that was produced by IT is taking away by T2. By 
moving messages on the path of T2, a new cycle is created again, involving q and 
p. If we suppose that the same thing happens again such as the extra buffer of s 
becomes full and that s and p becomes involved again in the another cycle then 
the system is deadlocked and we cannot do anything to solve it since we cannot 
erase any valid message. Thus we have to avoid to reach such a configuration 
dynamically. To do so, when the token circulation finds either a free buffer or 



detect a cycle, it does the reverse path in order to validate its path. Thus when 
the path is validated no other token circulation can use a buffer that is already 
in the validated path. Note that the token is now back to the initiator. To be 
sure that all the path of the token circulation is a correct path (it did not merge 
with another token circulation that was in the initial configuration) , the initiator 
sends back the token to confirm all the path. In another hand, since the starting 
configuration can be an arbitrary configuration, we may have in the system a 
path of a token circulation that forms a cycle. To detect and release such a 
situation, a value is added to the state of each buffer in the following manner: 
If the buffer Bi has the token with the value x, then when the next buffer Bi + i 
receive the token it will set it value at x + 1. Thus we are sure that in the case 
there is a cycle there will be two consecutive buffers B and B' having respectively 
x and x' as a value in the path of the cycle such as x ^ x'. Thus this kind of 
situation can be detected. 



Token Circulation Tl H 
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Fig. 2. Instance of a problem. 



3.2 Formal Description of the Solution 



In this section we first define the data and variables that are used for the de- 
scription of our algorithms. We then present the formal description of both the 
Token Circulation algorithm and the message forwarding algorithm. 
Character ' ? ' in the predicates and the algorithms means any value. 



Procedures 



• 



Next p (d): refers to the neighbour of p given by the routing tables for the 
destination d. 



• Deliver p (m): delivers the message m to the higher layer of p. 

• Choice(c): chooses a color for the message m which is different from the 
color of the message that are in the buffers connected to the one that 
will contain m. 



Variables 

• IN p (q): The input buffer of p associated to the link (p, q). 

• OUTp(q): The output buffer of p associated to the link (p, q). 

• EXT p : The Extra buffer of processor p. 

• Sp q i = (id, previous, next, phase, x): refers to the state of the input 
buffer of the process p on the link (p,q). id refers to the identity of the 
process that initiates the token circulation, previous is a pointer towards 
the output buffer from which the buffer pqi received the token (it refers 
to the output buffer of q on the link (q,p)). next is also a pointer that 
shows the next buffer that received the token from the input buffer of p 
on the link (p,q). phase G {S, V, F, C, E} defines the state of the token 
circulation to determine which phase is executed respectively (Search, 
Validation, Confirm, Escort or non of these "Clean" State), x is an in- 
teger which will be used in order to break incorrect cycles. 

• S pqo = (id, previous, next, phase, x): As for the input buffer, S pqo = 
(id, previous, next, phase, x) refers to the state of the output buffer of 
the process p connected to the link (p,q). The attributes have the same 
meaning as previously. 

• preVp qo : q' G N p such as S pq n — (id q >, q'po,pqo, S, ?) A id q > = min{id q >>, q" G 
N p A S pq »i = (id q n,q'po,pqo, S,l)}. 

• Small p : q G N p such as 3 q' G N p , S pq i — (id q ,l ,pq'o,F,x) A S pq > = 
(id q , X, q'pi, F, z) A X ^ pqi Az^i+lA id q = min{id q »,q" G N p A 
S pq "i = (id q »,?,pro,F,x') A S pro = (id q »,X',rpi,F,z') A X' ^ pq"i A 
z' ^ x' + 1. 

Predicates 

• NO - Tokenp. V q G N p , S pql = (-1, NULL, NULL, C, -I) A S pqo = 
(-1, ?, ?, ?) A S qpo = (-1, NULL, NULL, C, -I) 

We define a fair pointer that chooses the actions that will be performed on 
the output buffer of a processor p. (Generation of a message or an internal 
transmission) . 



Algorithm 1 Token circulation — Initiation and Transmission 

Token initiation 

Rl: Token p (q) A S pqo = (-1, NULL, NULL, C, -1) A S pqo = S vqi -> S^ : = 
(p, NULL, pqo, S, 0), 5 P g D := (p 5 P<?i) gpij 5, 1) 

Token transmission 

— Search phase 

• R2: 3 q, q £ 7V P , S qP o = (id,?, pqi, S,x) A IN p (q) = (m, d, c) A Next p (d) = q A S pg / / 
(id, ?,?,?,?) A SW / (id',?,?,V V F VE,?) A (SW / (id", ?,?,?, ?) A id" <= id) -> 
5 pg i :— (id, qpo, pq o, S, x + 1) 

• R3: 3q,g' £ 7V P , prev pqo — q A S pq /i — (id, qpo, pqo, S,x) A (S qp i 7^ (id,?,?,?,?) A 
S pgo / (id", ?, ?, V V F V F, ?) A S pqo / (id', ?, ?, ?, ?) A id' <= id A OUT p (q) ± e A 
OUT p (q) ± /iV q (p) -> S PQO := (id,pq'i,qpi,S,x + 1) 

— Validation phase 

• Initiation 

* R4: 3q, q' £ N p , prevpqo — q' A S^/^ — (id, q'po,pqo, S, x) A 5 pq o ^ (*d" 3 ?,?, V V 
F V F, ?) A S PQO / (id', ?, ?, ?, ?) A id' < id A S qp% = (id, X, ?, S, ?) A X ^ pqo A 
OUT p (q) ^e A OUT p (q) # /AT q (p) ->■ S pgo := (id, pq'i, gp2, V, x + 1) 

* R5: 3 q,q' £ 7V P . S gpo = (id,?,pqi, S,x) A IN p (q) = (m,d,c) A Next p (d) = q A 
5 pq / - (id,X,?,S,z) A X ^ pqi A EXT P = e A 5 pqi / (id',?,?, V V F V F, ?) A 
(5 pq i / (id",?, ?,?,?) A id" < id) -» S pq ; := (id,qpo,pq' ' o,V,x + 1) 

* R6: 3 q, q' £ 7V P , prev pqo = q A 5 pq /i = (id, qpo, pqo, S, x) A [(OUT p (q) = e V 
OUT p (q) = /ATg(p))] -)- S pgo := (id,pqi, NULL.V, x + 1) 

* R7: 3 q, g' £ -/V P , S qpo = (id,?, pqi, S,x) A IN p (q) = e -> S pqi : = 
(id,qpo,NULL,V y x + 1) 

• Transmission 

* R8: 3 q, q' £ 7V p , S pgo — (id, pq'i, qpi, 5, a;) A S qp i — (id, pqo, ?, V, x -\- 1) A x 7^ 1 A 
5 pq /^ 7^ (id, ?, pgo, F, x — 1) — s- S pqo :— (id, ?, gpi, V, x) 

* R9: 3 q.q' £ 7V p , S pg i — (id, qpo,pq' o, S, x) A S pq / — (id,pqi,?,V, x + 1) A 
S qpo 7^ («d, ? ,pqi, F, x — 1) — )■ 5 pq ^ :— (id, qpo, pq o, V, x) 

— Confirm phase 

• Initiation 

* RIO: 3 q £ 7V P , 5 pqo = (p, pqi, qpi, S, 1) A S pqi = (p, NULL, pqo., S, 0) A 
5 gp i = (p,pgo, ?, V, 2) -> 5 pqo :^ (p, pqi, qpi, F, 1), 5 pq i :^ (p, NU LL, pqo, F,Q) 

• Transmission 

* Rll: 3 q, q' £ W p , S'gpo — (id^ipqi, F, x) A S pg i — (id, qpo,pq' o,V, x + 1) — > 
'S'pgi :— (id, qpo, pg'o, F, a; + 1) 

* R12: 3 q,q' £ 7V p , prev pqo — q 1 A S , pg / i — (id, ?, pgo, F, a;) A 
•S'pgo — (id,pq'i, qpi, V, x + 1) — »■ 5 pqo :— (id,pq'i, qpi, F, x + 1) 

— Escort phase 

• Initiation 

* R13: 3 q £ 7V P , S pgi = (id, idle, pqo, F, 0) A S qpo = (id,?, pqi, F, x) A cc > 3 A 
•S'pgo — (*d, pqi, qpi, F, 1) A EXT P — e — > S'pqi :— (id, id/e, pqo, F, 0) 

* R14: Smallp — q A 3 q' £ 7V P , S PQ i — (id, qpo,pq o, F, x) A S pq / — (id, X, q pi, F, z) 
AX^pqiAz^x-\-lA EXT P = e A $ q" £ N p , (S pq ff t = (id 1 , NULL, Z, F, 0) A 
Sz — (id' , pq" i, ?, F, 1)) —> S pq i :— (id, qpo, pq o, E, x) 

* R15: 3 q £ 7V p , S qpo = (id, ?,pqi, F,x) A S pgi = (id, qpo, idle, V, x -\- 1) A IN p (q) = a 
—$■ S pq i :— (id, qpo, idle, E, x + 1) 

* R16: 3 q, q £ 7V p , S pq i — (id, qpo,pq o, F, x) A S pq / — (id,pqi,idle,V,x-\-l) A 
[OUT p (q) = e V OUT p (q) = IN q (p)} -> S pq , :^ (id, pqi, idle, E, x + 1) 

• Propagation 

* R17: 3q,q £ -/V p , S pgo = (id,? , qpi, F, x) A S qpi = (id, pqo, ?, E, x + 1 V 0) ->■ 
Sp q o ■— (id, ?, qpi, E, x) 

* R18: 3q, q' £ AT p , S pg i — (id, qpo, pq o, F, x) A S pq / — (id, pqi, q pi, E,x + 1) —> 
Spqi :— (id, qpo, pq o, E, x) 

* R19: 3q £ N p , S pqo — (id, pqi, qpi, F, 1) A S qp i — (id, idle, pqo, E,0) A S qp i — 
(id, pqo, ? , E, 2) — > S pqo '•— (id, pqi, qpi, E, 1) 



Algorithm 2 Token Circulation — Cleaning Phase and Correction 

— -Cleaning phase 

• Initiation 

* R20: 3 q G N p , S pqi = (id, NULL, pqo, E,0) A S pqo = (id, pqi, qpi, F, 1) -!■ S pqi : = 
(-1, NULL, NULL, C, -1) 

* R21: 3 q, q G N p , S pq /i — (id, q ' po, pqo, E, x) A S pqo — [id, X, qpi, E, z) A X ^ pq i 
-> S pqli := (-1, NULL, NULL, C, -1) 

* R22: 3 q G N p , S pqi = (id, qpo, NULL, E , x) A S qpo = (id,? ,pqi, E. x - 1) -^ 
S pqi := (-1, NULL, NULL, C, -1) 

* R23: 3 q, q G N p , S pqo — (id,pq i, qpi, E, x) A S pq i i — (id, q po,pqo, E, x — 1) — » 
S pqli := (-1, NULL, NULL, C,-l) 

• Propagation 

* R24: 3 q G iV p , S pqo = (id, X, qpi, E, x) A Si / (id,? , pqo, F, x - 1) A [(S, p i = 
(id',?,?,?,?) A id # id') V S, p i = (-1, NULL, NULL, C, -1)] -*■ Sp,j : = 
(-1, NULL, NULL, C, -1) 

* R25: 3 q £ iV p , 5 pqi = (id, qpo, pq o, E, x) S qpo jt (id,? ,pqi, F,x - 1) A [(S pq / = 
(id',?,?,?,?) A id jt id') V 5 p9 / = (-1, NULL, NULL, C, -1)] -> S pgo : = 
(-1, NULL, NULL, C,-l) 

— Correction rules 

• Freeze Cleaning 

* Initiation 

- R26: 3q, q' G N p , S pqo = (id,pq'i,qpi, S V V V F, ?) A [S pqli = 
(-1, NULL, NULL, C,-l) V (S pg , s = (id',?,?,?,?) A id' ^ id) V (S p9 ,< = 
(id, ?, M, ?, ?) A M # p 9 o)] -» S pqo := (id, pg'i, qpi, G, ?) 

- R27: 3g, q' G JV P , S pgi = (id, qpo,pq' o, ? , ?) A (S, po = (-1, NULL, NULL, C, -1) 

V (S 9po = (id',?,?,?,?) A id' ^ id)) -> S pqi := (id, qpo, pq' o, G,?) 

- R28: S p9i = (p, NULL, pqo, ?, x) A x > -> S p9i := (p,NULL, pqo, G,x) 

- R29: 3 q, q £ iV p , S pq i — (id, ? , pq o, ? , x) A S pqo — (id, pq i, qpi, ? , z) A z 7^ x + 1 
— > Spgi :— (id,?,pq'o,G,x) 

- R30: 3 q £ AT p , S p9 i — (id, qpo, pqo, ?, a;) A S qpo — (id, ? ,pqi, ? , z) A z 7^ a; + 1 — > 
S P qi :— (id,qpo,pqo,G,x) 

- R31: 3 g G ATp, [(Sp,j = (id,?,qpi,S,x) A S 9P » = (id, pqo, ?, F V E, a; + 1)) V 
(S pqo = (id,?, qpi, F,x) A S qp i = (id, pqo, ?, S, x + 1)) V (S P9C) = (id,? , qpi,V,x) A 
S 9pi = (id, pqo, ?, F V S, x + 1))] -> S pgo := (id, ?, qpi, G, x) 

- R32: 3 q,q' G iV p , [(S p9i = (id, ?, pq' o, S, x) A S pq , = (id, pqi, ? , F V E , x + 1)) V 
(S pqi - (id,?,pq'o,F,x) A S pq i - (id, pqi,?, S, x + 1)) V (S pqi = (id ,? , pq' o ,V , x) 
A S pqlo = (id,pqo,?,EV S,x + 1))] -> S pgi := (id, ? ,pq' o, G, x) 

* Propagation 

- R33:3 q,q G iV p , S„ po = (id,? , pqi, G, ?) A S p ,i = (id, qpo, pq'o, S V V V F V E, ?) 
— > S pq i :— (id,qpo,pq o,G,?) 

- R34: 3 q, q' G iV p , prev pqo = q A S p9 ; = (id,?, pq' i, G,?) A 
S pg , = (id, qpo, q'pi, S V V V F V F, ?) -» S pq / := (id, pqi, q'pi, G, ?) 

* Cleaning 

R35: 3q,q' G iV p , S pq i = (id, qpo, pq' o, G, x) A [S pq i = 

(-1, NULL, NULL, C, -1) V (S p ,; = (id',?,?,?,?) A id' jt id) V 
(S p9 / = (id, ?, qpi, G, z) -¥ S pqi ■- (-1, NULL, NULL, C, -1) 

- R36: 3q,q' G N p , S pqo - (id, pq' i, qpi, G, x) A [S qpi = (-1, NULL, NULL, C, -1) 

V (S qpz = (id',?,?,?,?) A id' 5^ id) V (S, p , = (id,pqo,?,G, z) -¥ 
S pqo ■- (-1, NULL, NULL, C, -1) 

• R37: 3 q, q' G A^ p , S pq i — (id, ? ,pq o, G, x) A S pq r — (id, pqi, q'pi, G, z) A z ^ x + 1 — > 
S pqi := (-1, NULL, NULL, C, -1) 

• R38: 3 q G iV p , S pq i — (id, qpo, pqo, G, x) A S qpo — (id,? , pqi, G, z) A z 7^ x + 1 — > 
S pqi :=(-!, NULL, NULL, C, -I) 

• R39: Token p (q) A S pqi = (p, ?,?,?,?) -> Token p (q) := false 

• R40: 3 q, q' G iV p , S pqi = (id, qpo, pq'o, F, x) A S pq i = (id, X, q'pi, S VV, z) A z ^- x + 1 
-> S p ,i := (-1, NULL, NULL, C, -1) 

< R41: 3 q G JV p , S pqi = (id, qpo, NU LL, S V V V F, cc) A IN p (q) =£ e -> S pqi := 
(-1, NULL, NULL, C, -1) 

« R42: 3 q,q' G iV p , S pqo = (id, pq'i, NULL, S V V V F, x) A OUT p (q) # e -!■ S p , := 
(-1, NULL, NULL, C, -1) 

• R43 3 q G JV P , Sp qo = (iii,?, W !,VvF,x) A [(S 9pi = (id',?,?,?,?) A id =£ id') V 
S, pi = (-1, NULL, NULL, C, -1)] -> S pqo := (-1, NULL, NULL, C, -1) 

< R44 3 q G JV P1 S p<! j = (id, qpo, pq'o, V V F V F, x) A [(S pq i a = (id',?, ?, ?, ?) A id ^ id') 
V S /„ = (-1, NULL, NULL, C, -1)] -s- S p , := (-1, NULL, NULL, C, -1) 



Algorithm 3 Message Forwarding 



Message generation (For every processor) 

R'l: Request,, A Next p (d) = q A [OUT p {q) = £ V OUT p {q) = IN q {p)} A NO - Token -> 
OUT p (q) : — (m, d, choice(c)), Request p :— false. 

Message consumption (For every processor) 

R'2: 3g e N p , IN p (q) = (m.d.c) A d = p A OUT q (p) =£ IN p (q) -> deliver p (m), 

IN p (q):=OUT q (p). 

Internal transmission 

R'3: 3g,g' G N p , IN p (q) = (m.d.c) Ad/pA Next p (d) = q A q jt q A [OUT p (q) = e V 
OUT p (q') = /JV g ,(p)] A OUT q (p) # IJVp(g) A WO - Toten -> OUT p (q') := (m,d,choice(c)), 
IN p (q) :=OUT q (p). 

R'4: 3 9 ,g' e JV P , IN p (q') = (m,d,c) A OUT q ,(p) # IN p (q') A [Ot/T p (g) = e V 
OUT p (q) = 7JV,(p)] A S pqo = (id, pq' i, qpi, E,x + 1) A S pg /j = (id, q' po, pqo, F, x) -> 
OUT p (q) := (m,d,choice(c)), IN p (q') := OUT q , (p) 

Message transmission from q to p 

R'5: IN p (q) = e A OUT q (p) = (m, d, c) A g # d A NO - Tofceri -> IN p (q) := OUT q (p). 

R'6: 3g e JV P , IN p (q) - e A OUT q (p) - (to, d, c) A 9 5^ d A S pqi = (id, qpo,?, E, x + 1) A 
S, po = (id,?,pqi,E,x) -> IJVp(g) := OUTq(p) 

Erasing a message after its transmission 

R'7: 3g e JV P , OUT p (q) = IN q (p) A (Vg' £ JV P \ {-}}, /JV p (g') = e V 
(IN p (q) = (ro, d, c) A Next p (d) # 3)) A AfO - Totem -> OUT p (q) := e 

Erasing a message after its transmission (For the leaf processors) 

R'8: JVp = {g} A OUT p (q) = JJV,(p) A (IN p (q) = e V (IN p (q) = (to, d, c) A Next p (d) ^ q)) A 

WO - Totem -> OUT p (q) := e 

Road change 

R'9: 3 g e JV P , 7JV p (g) = (to, d, c) A Next p (d) = q A OUT p (q) = e V OUT p (q) = IN q (p) -> 

OUT p (q) ~ IN p (q), IN p (q) := OUT q (p) 

R'10: 3 g 6 ATp, 7JV p (g) = (m, d, c) A Next p (d) = q A OUT p (q) =£ e A OUT p (q) jt IN q (p) A 
73XT P = e A J g' £ JV P , S pQ ; 4 = (id, ?,pg'o, ?, 0) -»■ Token p (q) := true 

R'll: 3 g G JV P , Sp,ji = (id, idle, pqo, F, 0) A S qpo = (id,? ,pqi, F,x) A x > 3 A 
S p ,o = (id,pqi, qpi, F, 1) A 73XT P = e -)■ 73XT P := IN p (q), IN p (q) := OUTq(p) 

R'12: Sraall p — g A 3 g' G JV P , 5 pq i — (id, qpo,pq' o, F, x) A S / — (id, X, q' pi, F, z) A 
x/p»tAz^i + lA S, po = (id,?,pqi,F,x- 1) A J g" £ JV P , (S p9 « ( = (id' , NULL, Z, F,0) 
A Sz = (id',pq"i,?,F,l)) -> BJCT P := IN p (q), IN p (q) := OUTq(p) 

R'13: 3 g G JVp Sp„j = (id, NULL, pqo, E,0) A S p <, = (id,pqi, qpi, F, 1) A 
S qpi = (id,pqo,?,E,2) A 73XT P 5^ e A (OUT p (q) = e V OUT p (q) = IN q (p)) -> 
OUT p (q) := EXT P , EXT P := e 

R'14: 3 (J, ij' S JVp Sp^/; = (id,q'po,pqo, E,x) A Sp,„ = (id, X, qpi, F, z) l\ X ^ pq' i A 
z # x + 1 A Sgpj = (id,pqo,?,E,z + 1) A BXT P # e A (OUT p (q) = e V OUT p (q) = IN q (p)) 
-> OUT p (q) ■= EXT P , EXT P := e 

Correction Rules 

R'15: JTXTp 5^ e A (JVO - Totem A (V g e JV P , S pqi 7^ (id, qpo,'!, E)) A (3 g 
£ JVp, Sp qi = (id, NULL, pqo, E,0) A S p , = (id,pqi, qpi, E,l) A OUT p (q) ^ e A 
OUT p (q) ^ JJV,(p)) -»■ BXTp := e 

R'16: JTXTp # e A (JVO - Totem A (V g £ JV P , Sp„j # (id, qpo,? , E)) A (3 g, g' £ JV P , 
Sp g /i = (id, ?,pgo, 73, x) A S pgo = (id, X, gpi, 73, z) A X 5^ pg'i A z 5^ x + 1 A OUT p (q) jt e A 
OUT p (q) jt IN q (p)) -> EXT P := e 

R'17: Token p (q) = true A IN p (q) = e V IN p (q) = (m,d,c) A Next/^d) ^ q -> 
Token p (q) — false 



3.3 Proof of correctness 

We prove in this section the correctness of our algorithm. The idea of the proofs 
is the following: we first show that no valid message is deleted from the system 
unless it is delivered to its destination. We then show that each buffer is infinitely 
often free, thus neither deadlocks nor starvation appear in the system. We finally 
show that every valid message is delivered to its destination once and only once 
in a finite time. Before detailing the proofs, let define some notions that will be 
used later. 

Definition 1. Let B\ and B 2 be two buffers and p, q and q' be processors in the 
network such that one of those properties holds: 

- 3 p,q,q' such as B x = IN p (q) AB 2 = OUT p (q') 

- 3p,q such as B x = OUT p (q) A B 2 = IN q (p) 

B2 is called the successor of Bl denoted by B\ h^ B2 if and only if Sb x = 
(id, ?, B 2 , ?, x) A S B2 = (id, Bi, ?, ?, x + 1) 

Definition 2. A sequence of k buffers B\ n> B 2 <— > ... 1— > Bk starting from B\ 
is called an abnormal sequence if the following property holds: 

S Bl = (id, ?, ?, ?, ?) A (Bi = IN p (q) V Bi = OUT p (q)) A id ± p 

A buffer B is said to be cleared if S B = (-1, NULL, NULL, C, -1). In the 
same manner, a sequence is said to be cleared, if all the buffers part of it becomes 
cleared in a finite time. 

Let us state the following lemma: 

Lemma 1 If the configuration contains an abnormal sequence S\ of buffers 
Bi M> B 2 <— > ... i-4 Bk, then Si will be cleared in a finite time. 

Proof. Since Si is an abnormal sequence. There is one processor p that 
has sent a token which it did not receive from any other processor (p is not 
the initiator). This processor is the one with the buffer Bi. Note that p will 
be able to detect such a situation and either R26 or R27 will be enabled on 
p. When p executes either R2Q or R27 the Freeze cleaning phase is initiated, 
thus, Sbj = (id,NULL,B 2 ,G,x). Either R33 or i?34 becomes enabled on the 
process that has B 2 as a buffer. Once one of these two rules is executed Sb 2 = 
(id, Bi, B3, G, ?), B3 will set also its state to the Freeze cleaning phase, and so 
on. Thus all the buffers that are in the sequence Bi 1— > B 2 n> ... n> Bk will be in 
the freeze phase G. Note that on the process p' that has Bk as a buffer (note that 
Bk is the last buffer of the sequence), either i?35 or B36 is enabled onp'. Once p' 
executes one of these rules Bk is cleared. i?35 or R36 becomes then enabled on the 
process that has Bk-i as a buffer. Thus when one of these rules is executed B^_i 
is cleared as well and so on. Thus we are sure that after a finite time each buffer 
that is in Si will clear its state and the Lemma holds. Note that the sequence 
Si can be broken (another token circulation with a smallest identifier can use 
one of the buffer of Si ) . Note that in this case Si is divided in two sub abnormal 
sequence. Each sub abnormal sequence will behave on its own. Thus the buffers 
in each sub abnormal sequence will be cleared in a finite time. Observe that if 



in the sequence Ss k — (id, -Bfc-i, B\, ?, z) and Sb ± = (*<i, -Bfc, -B2, ?,x) then we 
are sure that z 7^ X + 1. In this case too, the processor having B\ as a buffer 
will be able to detect such a situation and initiates the freeze cleaning phase as 
previously. Thus the lemma holds. □ 

Let p, q and q' be processors such as q, q' £ N p , we state the following 
Lemma: 

Lemma 2 If a valid message m is copied in EXT p from IN p (q) in order to be 
copied later in OUT p (q'), then when S pq i — (id, ?, ?, E, ?), EXT p is free. 

Proof. Since the message m is copied in EXT p , m is in the wrong direction. 
IN p (q) containing m is part of a complete token circulation T i.e., a token 
circulation that validated and confirmed all its path (Recall that no message 
can be generated in the presence of a token circulation (see Rule R'l) and, if an 
abnormal token circulation reaches IN p (q) after the generation of the message 
m, we are sure that the path of such a token will never be confirmed moreover 
all the buffers part of it will clear their state in a finite time (refer to Lemma 
[1}). To simplify the explanation let us define T as follow: T = Bi h> B2 ^ 
... h-> Bk- Note that IN p (q) (mentioned in the lemma) can be either B\ (in 
the case of a full-cycle) or Bk (in the case of a sub-cycle). In the following we 
will consider only the case of a full-cycle (the same reasoning holds for the sub- 
cycle case). We show that there is a synchrony between the forwarding and the 
token circulation algorithms. When the token circulation confirmed all its path 
(all the buffer part of T have their State attribute set at E), R'll and i?13 
becomes enabled on p. Recall that in this case p executes both of them, thus m 
is copied in EXT p , B\ = Bk and Sb ± = (id, NULL, B 2 , E,0) (Bk becomes a 
free buffer). R17 becomes enabled on the processor with the buffer B^. When the 
rule is executed Sb,. = (id, Bk-i, B\,E, x). Observe that Bk is an output buffer 
whereas Bk-\ which is an input buffer. Both R'A and R18 become enabled on 
the processor with the two buffers Bk and B^-\. When both rules are executed 
Ss k _ 1 = (id, .Bfc-2, Bk, E, x) and -Bfc-2 = Bfe_i. Note that the same situation 
as the first one appear. We can observe that when an output buffer B part of 
T is free with the state Sb = (id, ? ', ?, F, z), R17 is enabled on the processor p 1 
with the buffer B, thus the state of B will be set to Sb = (id, ?, ?, E, z) and 
notice that B remains free. Thus on p' two rules will be enabled (the internal 
transmission (i?'4) and the propagation of the escort phase (R18)), when both 
are executed we retrieve the same situation with another empty output buffer, 
and so on. Hence we are sure that on the processor p, R'13 and R19 will be 
enabled at the same time. When both rules are executed, EXT p is free and 
Sb 2 = (id, Bi,B 3 , E, 1) where B 2 refers to OUT p (q) and the lemma holds. 

D 

We can now detect in some cases if the message in the extra buffer is invalid 
(it was in the initial configuration). Note that the algorithm deletes a message 
only in such cases (when we are sure that the message in the extra buffer is 
invalid), refer to Rules i?'15 and R'16. Thus we have the following Theorem: 

Theorem 1. No valid message is deleted from the system unless it is delivered 
to its destination. 



Proof. The proof is by contradiction: we first suppose that there is a mes- 
sage m that is deleted without being delivered to its destination. 

— By construction of R'3 and R'A, this cannot be a result of an internal forward- 
ing since the message m is first of copied in OUT p (q) before being erased from 
IN p (q). Note that these two rules are enabled only if OUTp^q) = IN q (p) 
or OUTp(q) — e. Hence when the message m is copied in the OUT p (q) no 
message is deleted. 

— By the construction of Rule R'5 and R'6, the message is only copied in 
INp(q) and not deleted from the OUT q (p). Note that IN p (q) is empty. Thus 
no message is erased in this case. 

— By the construction of rule R'7 and R'8, the message in OUT p (q) is deleted. 
However note that in this case OUT p (q) = IN q (p). Thus there is still a copy 
in the system of the message erased. 

— By the construction of rule R'9, R'13 and R'\A. The message is first copied 
in OUT p (q) (note that OUT p {q) is in this case empty) before being erased. 

— The same holds for R'll and R'12, the message in IN p (q) is first copied in 
EXT P (note that EXT P is in this case empty), before being erased. Thus 
there is still a copy in the system of such a message. 

— Concerning R'15 and R'W, according to Lemma[5J If one of these rules (R'15 
or R'16) is enabled in p then we are sure that the message in EXT P is an 
invalid message. Thus when the processor p executes on of them, no valid 
message is deleted. 



We can deduce from all the cases above that no valid message is deleted unless 
it is delivered to its destination, hence the lemma holds. 

□ 

We now show in Lemma [3] that the extra buffer of any processor p cannot 
be infinitely continuously busy (Recall that the extra buffer is used to solve the 
problem of deadlocks) . 

Lemma 3 If the extra buffer of the processor p (EXT p ) contains a message, 
then this buffer becomes free after a finite time. 

Proof. Suppose that the extra buffer contains a message. The cases below 
are possible: 

1. There is no token circulation including an input buffer of p. In this case the 
message that is in the extra buffer EXT p is deleted by the processor p by 
executing either R'15 or R'16 and the lemma holds. 

2. There is no q, q' £ N p such that either (i) S pq i = (id, NULL, pqo, State, 0) 
and S pqo — (id, pqi,qpi, State', 1) or, (ii) S pq i — (id, qpo,pq'o, State, x) and 
S pq io — (id, X, q'pi, State' , z) and z ^ x+1 hold. In this case too the message 
that is in the extra buffer EXT p is deleted by the processor p by executing 
either R'15 or R'16 and the lemma holds. Observe that if either (i) or (ii) 
holds for q, q' £ N p such that State ^ E and State' ^ F then in this case 
too the message in EXT p is deleted. 



3. if there exists q, q' 6 N p such that either (i) S pq i = (id,NULL,pqo,E,0) 
and S pqo — (id,pqi,qpi,F,l) or, (ii) S pq i = (id,qpo,pq'o,E,x)andSpq> = 
(id, X,q'pi, F, z) and z/x + 1 holds then the following two sub-cases are 
possible: 

— The token circulation is an abnormal sequence. In this case, we are sure 
that all the buffers part of it will clear their state (refer to Lemma [T]). 
Thus we retrieve case [U 

— The token circulation is a valid token circulation. In this case we are sure 
that the state of OUT p (q) (resp, OUT p (q')) will be set at (id, ?, ?, E, ?). 
Thus if OUT p (q) (resp OUT p (q')) is free then the message in EXT p is 
copied in it (refer to Rules i?'13 and R'lA). If it is not free the message 
in EXT p is deleted (refer to R'15 and R'16), 

From the cases above, we can deduce that if EXT p is occupied then it will be 
cleared in a finite time. □ 



Lemma 4 If there is a Token Circulation that validates all its sequence, then 
all the buffers part of it will clear their state in a finite time. 

Proof. Let refer to the Token Circulation as T = B\ (->• P>2 <->• ... i-» Bk- 
Since the token circulation validates all its path then either it found a free buffer 
or detects a cycle. Note that (i) if its' non of these cases hold then the last buffer 
of the sequence Bk will clear its state (RA\ or RA2 is executed). Bk-i then does 
the same and so on. Otherwise, (ii) the Confirm phase is initiated by the initiator 
of T and we can easily show that all the buffers part of T will update their state 
to the confirm phase in a finite time. The escort phase is then initiated by either 
the initiator of T (in the case of a full-cycle) or by the processor that has Bk 
as a buffer. Observe that the escort phase progresses in the reverse sequence of 
T when it reaches the initiator (in the case of a full cycle) the initiator initiates 
the cleaning phase by clearing B\ (In the case of a sub-cycle the processor 
that detects the sub-cycle is the one that initiates the cleaning phase when its 
corresponding output buffer updates its state to the escort phase. For instance, 
in Figure [2] (c), the processor q detects the cycle, q initiates the cleaning phase 
when it updates the state of OUT q (r) to (id,? ,rqi, E,?)). In the same manner 
Bk-i will clear its state and so on. Thus in this case too we are sure that all the 
buffers part of T will clear their state in a finite time. Observe that in the case 
T found a free buffer the cleaning phase is initiated by the processor with the 
buffer Bk- Bk-i clears then its state and so on. Thus we are sure that all the 
buffers part of T will clear their state in a finite time and the lemma holds. □ 



Lemma 5 In the case where Token p (q) — true, it will be set at false in a finite 
time. 

Proof. Note that in the case Token p (q) — true and the rule that allows the 
initiation of the token circulation is enabled, Token p (q) will be set at false by 
the token circulation algorithm when the this rule is executed. Otherwise, the 
two cases below are possible: 



— Sp q i = (id, ?, ?, ?, ?), in this case Token p (q) will be set at false by the Token 
Circulation algorithm by executing R39 (Note that R39 is enabled on p and 
the daemon is weakly fair). 

— Spqi = (— 1, NULL, NULL, C, —1). In the case the next processor by which 
the message that is in IN p (q) have to pass to reach the destination is q 
then the rule that allows the initiation of the token circulation is enabled on 
p. Thus, Tokerip(q) will be set at false by the token circulation algorithm 
when the this rule is executed. Otherwise, Token p (q) will be set at false by 
executing R'17 that is enabled on p. 

From the cases above we can deduct that in the case Token p (q) = true, it 
will be set at false in a finite time and the lemma holds. □ 



Lemma 6 // there is a processor that wants to generate a token circulation, it 
will be able to do it in a finite time. 

Proof. From Lemma [5] we know that if Token p is true then it will be set at 
false in a finite time. From Lemma |3] if EXT p is occupied, then it will be cleared 
in a finite time. From Lemma [4] and Lemma Q] we know that if there is a token 
circulation that is executed all the buffers part of it will clear their state in a 
finite time. Thus when p wants to generate a token circulation it will be able to 
do it in a finite time. □ 



Lemma 7 // there are some Token Circulations that are initiated then at least 
one of them will validate all its path. 

Proof. Let us focus on the token circulation that has the smallest id (Let this 
token be Tl). When such a token circulation is initiated, the only things that 
can stop its progression is the presence on the path of another token circulation 
T2 that is in the Valid phase. Thus the following cases are then possible: 

1. i) T2 is a correct token circulation. In this case two sub cases are possible as 
follow: i) all the path of T2 has been validated. No other token circulation 
can break T2. Thus according to Lemma HI we are sure that the state of all 
the buffers of the path will be clean in a finite time. Thus Tl can continue its 
progression, ii) There is another token circulation T3 that cut T2. Note that 
in this case there is a part of the path that has been broken. An abnormal 
sequence is then created (Note that the buffers that were part of Tl that are 
in the valid phase are part of the abnormal sequence) . According to Lemma 
[1] the state of the buffers of the sequence will cleared. Thus Tl can continue 
its progression. 

2. ii) T2 is not a correct token circulation. In this case T2 is an abnormal 
sequence. In this case according to Lemmas [1] The state of the buffers part 
of T2 will be cleared in a finite time. Thus Tl can continue its progression. 
Note that T2 can behave as a valid token circulation. In this case we retrieve 
case[U 

In both cases Tl continues its progression. Thus we are sure that Tl will 
be able to reach the last buffer Bi such as Bi is either empty or it wants 



to send the token to a buffer that is already in the path of Tl. Note that 
on the processor that contains Bi either R4 or R5 or R6 or R7 are enabled. 
The second phase is then initiated (the state of Bi will be valid). It is easy 
to prove by induction that all the buffer on the path of Tl will be validated 
since that we are sure that there is no other token circulation that can break 
Tl (Recall that Tl has the smallest id). Thus the lemma holds. 

□ 

We can then deduce that at least one message will undergo a route change. 
The next lemma follows: 

Lemma 8 When the routing tables are stabilized all the messages will be in a 
suitable buffer in a finite time. 

Proof. Note that when the routing tables are stabilized, some messages may 
be on the wrong direction, however, we are sure that the number of such messages 
will never increase since both the generation and the routing of messages is 
always done in the right buffer (Recall that the routing tables are stabilized). In 
another hand according to Lcmma[3l if the extra buffer of p (EXT p ) is occupied, 
it will be free in a finite time. Suppose that p is the processor that has an input 
buffer that contains a message m that is not in a suitable buffer. This process will 
initiates a token circulation. According to Lemma [7] There is at least one token 
circulation that will finish its execution (Suppose that this token circulation is 
the one that was initiated by p). Thus we are sure that the output buffer of 
p (the next destination of m) will be free in a finite time (refer to Lemma [2j 
Thus the message in m will be copied in the free output buffer. Note that once 
it is copied in the corresponding output buffer, it becomes in a suitable buffer. 
Hence the number of the messages that are not in a suitable buffer decreases at 
each time. Thus we are sure that at the end all the messages will be in the right 
direction and hence in a suitable buffer and the lemma holds. □ 



Lemma 9 When the routing tables are stabilized and all the messages are in 
suitable buffer, no Token circulation is initiated. 

Proof. According to LemmaO For any q € N p Token p (q) will be set at false 
in a finite time. Note that the only rule that set Token pq at true is i?'10. However 
R'lO is never enabled since all the messages on the system are in suitable buffer 
and since the routing tables are correct (all messages are generated and routed 
in suitable buffers as well). Thus the lemma holds. □ 

The fair pointer mechanism cannot be disturbed anymore by the token cir- 
culations. Note that our buffer graph is a DAG when the routing tables are 
stabilized Thus: 

Lemma 10 All the messages progress in the system. 

Proof. 

In order to prove the lemma, it is sufficient to prove that all the buffers are 
continuously free. Note that if 3 q e N p such as , IN p (q) is free then if there is 



a message in OUT q (p), this message is automatically copied in IN p (q) and thus 
OUT q (p) becomes free. Hence it is sufficient to prove that the input buffers are 
free in a finite time. To do so, let's prove that V p £ I, when there is a message 
in INp(q), this message is deleted from INp(q) in a finite time (q £ N p ). 

Recall that after the stabilization of the routing tables all the messages will 
be in the right direction and no token circulation is initiated (refer to Lemma [5] 
and in]) ■ Let consider the system after the stabilization of the routing tables and 
when all the messages are in the right direction. Let consider the message to that 
is in the input buffer of the processor p, referred to as B\. Let B\, B 2 , B 3 , ..., Bk 
the be the sequence of buffers starting from £?isuch as Bi = m' and Bi + \ is the 
next buffer by which to' should pass by to reach its destination. Note that Bi is 
an input buffer when i is odd. In the worst case V 1 < i < k all the buffers are 
full and Bk is the input buffer of a leaf processor that we will call po (Recall the 
all the messages are on the right direction). Note that the input-buffers in the 
sequence B\ , Bi , B% , . . . , Bk are all at an even distance from the input buffer Bk ■ 
Let define 5 as the distance between the input buffer of the processor p and the 
input buffer of processor po (Bk). The lemma is proved by induction on 5. We 
define for this purpose the following predicate P$ : If there is a message m in Bi 
such as Bi is an input buffer and at distance 6 from the input buffer Bk then 
one of these two cases happens: 

— to is consumed and hence delivered to its destination. 

— to is deleted from the input buffer and copied in .Bj+i (which is an output 
buffer). 

Initialization. Let's prove the result for Bk on p . Suppose that there is a 
message m in Bk- In this case we are sure that po is the destination of the 
message (otherwise the message to is in the wrong direction since po is a leaf 
processor). Thus, in this case, since the daemon is weakly fair and since R'2 keep 
being enabled on p$ then R'2 will be executed in a finite time and the message 
m in Bk is consumed. Thus Po is true. 

Induction, let S 1. We assume that P2& is true and we prove that P2S+2 is 
true as well (Recall that the input buffers are at an even distance from the input 
buffer Bk that in the processor p ). Let Bi on be the input buffer of p that is 
at distance 25 from Bk and -Bi_2 the one that is on p' being at distance 26 + 2 
from Bk containing the message to. In the case where the destination of to is p' 
then it will be consumed in a finite time (the daemon is weakly fair and R'2 keep 
being enabled on p'. Thus p' will execute R'2 in a finite time). Hence P28+2 is in 
this case true. In the other case (the destination of m is different from p'), since 
P28 is true then if there is a message to in Bi then we are sure that this message 
will be either consumed or copied in B i+ i. Thus Bi becomes a free buffer. The 
cases bellow are possible according to the rule that is executed on Bj_i: 

1. R'3 is executed. In this case one message that is in an input buffer of p is 
copied in -B^-i. However, since the pointer on Bi_i is fair, we are sure that 
the message to in Bi-2 will be copied in Bi_\. Thus Bi-2 will be free in a 
finite time and the lemma holds. 



2. a message m' is generated in Bi^\. However since P25 is true Bi becomes 
free in a finite time thus m' will be copied in Bi in a finite time. Thus -Bi_i 
becomes free. Nevertheless, since one message has been generated in the 
previous step, we are sure that R'3 will be the one that will be executed. 
Thus we retrieve Case [U 

□ 

Lemma 11 Any message can be generated in a finite time under a weakly fair 
daemon. 

Proof. According to Lemma |H1 no token is initiated when the routing tables 
are stabilized and when all the messages are in suitable buffers, thus the fair 
pointer mechanism cannot be disturbed by the token circulation anymore. Note 
that since the routing tables are stabilized and since the buffer graph is a DAG 
no deadlocks happens. Thus all the messages progress in the system. Suppose 
that the processor p wants to generate a message. Recall that the generation of 
a message m for the destination d is always done in the output buffer of the 
processor p connected to the link (p,q) such as Next p (d) — q. Two cases are 
possible: 

1. OUT p (q) — e . In this case, the processor executes either R'l or R'3 in a 
finite time, the result of this execution depends on the value of the pointer. 
Two cases are possible: 

— the pointer refers to R'l. Then p executes R'l and hence it generates a 
message. Thus we obtain the result. 

— the pointer refers to R'3. Then p executes R'3 in a finite time. Hence 
OUT p (q) ^ e and we retrieve case[2j Note that the fairness of the pointer 
guarantees us that this case cannot appear infinitely. 

2. OUT p (q) ^ e. Since all the messages move gradually in the buffer graph we 
are sure that OUT p (q) will be free in a finite time and we retrieve [1] 

We can deduct that every processor can generate a message in a finite time. □ 

We can now state the following Theorem: 

Theorem 2. Neither deadlock nor starvation situations appear in the system. 

Proof. According to Lemma 1101 All the messages progress in the system. 
Thus we are sure that there is no message that stays locked on one buffer, in 
another hand according to Lemma [TT1 every processors will be able to generate 
a message. Hence the Theorem holds. □ 

Lemma 12 The forwarding protocol never duplicates a valid message even if 
the routing algorithm runs simultaneously. 

Proof. Let consider the message m. The cases below are possible: 

— m is in EXT P . m is then either deleted or copied in OUT p (q). Since this 
operation is a local operation (the copy is done between two buffer of the 
same processor) m is copied in the new buffer and deleted from the previous 
one in a sequential manner. 



— m is in IN p (q). The cases are then possible: 

• ?n is consumed (R'2 is executed). The message m is deleted since a new 
value overwrites it. 

• m is copied in the extra buffer (i?'ll or R'12 is executed). The message 
m is copied in the extra buffer and deleted from the input buffer since 
in both cases a new value overwrites it. 

• m is copied in the output buffer (R'3 or R'4 is executed). Note that this 
operation is a local operation. Thus m is copied in the output buffer an 
deleted from the input buffer (a new value overwrites it). 

— m is in OUT p (q). m is then copied in the input buffer of the processor 
q (IN q (p)). Hence two copies are in the system. However m in IN q (p) is 
neither consumed nor transmitted unless the copy in OUT p (q) is deleted 
(see Rules R'2, R'3 and R'i). 

From the cases above we can deduce that no message is duplicated in the 

system. □ 

Theorem 3. The proposed algorithm (Algorithms 1, 2 and 3) is a snap-stabilizing 
message forwarding algorithm (satisfying SP) under a weakly fair daemon. 

Proof. From Theorem [T] no valid message is deleted. From Theorem [2] 
There is no deadlocks in the system and all the processors are able to generates 
messages in a finite time. From Lemma [T^l no message is duplicated. Hence, the 
theorem holds. 

□ 



4 Conclusion 

In this paper, we presented the first snap-stabilizing message forwarding protocol 
on trees that uses a number of buffers per node being independent of any global 
parameter. Our protocol uses only 4 buffers per link and an extra one per node. 
This is a preliminary version to get a solution that tolerates topology changes 
provided that the topology remains a tree. 
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